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Abstract 


The  limits  of  a class  of  primal  and  dual  solution  trajectories  associated  with  the 
Sequential  Unconstrained  Minimization  Technique  (SUMT)  based  on  the  loga- 
rithmic barrier  function  are  investigated  for  convex  programming  problems  in  the 
presence  of  multiple  optima  and  degeneracy  conditions.  For  linear  programming 
problems,  such  limits  can  be  characterized  as  analytic  centers  of,  loosely  speaking, 
optimality  regions.  Of  interest  here  are  generalizations  of  those  results. 

A class  of  convex  programming  problems,  identified  by  what  is  termed  the  “rank- 
integrity  property”,  is  introduced  for  which  primal  trajectory  limits  can  be  char- 
acterized in  analogy  to  the  linear  case  and  without  differentiability  conditions. 
This  class  of  problems  contains  linear  and  convex  quadratic  programming  prob- 
lems as  strict  subsets.  Similar,  albeit  more  restricted,  results  are  obtained  for  dual 
trajectories  defined  in  the  differentiable  case. 

Keywords:  analytic  center,  barrier  function,  convex  programming,  degeneracy, 
interior  point  method,  optimization,  rank-integrity,  SUMT,  trajectory. 


1.  Introduction 

Consider  the  following  “ primal ” convex  programming  problem  defined  on  a convex 
domain  C C iU1, 

, . Minimize  f{x)  subject  to 

gi(x)  > 0,  for  i = 1 ,...,m,  x G Rn: 

where  the  objective  function  f(x)  is  continuous  and  convex  on  (7,  and  the  con- 
straint functions  gz(x),  i = 1 ,...,m,  are  continuous  and  concave  on  C. 

If  an  optimal  solution  x*  exists,  let  t*  = f(x*)  denote  the  minimum  value  of  the 
programming  problem.  The  sets 

XF  = {x:  g,(x)  > 0,  i = 1, ....  m)  C C,  X*  = {x  : f(x)  =f,  xe  XF}  C XF 

of  feasible  and  optimal  solution  will  here  be  referred  to  as  “feasibility  region ” and 
“optimality  region ”,  respectively.  Both  are  closed  convex  sets.  We  will  restrict 
our  analysis  to  convex  programming  problems  for  which 

(1.2)  the  optimality  region  X*  is  nonempty,  bounded  and,  consequently, 
compact, 
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and  we  are  particularly  interested  in  those  cases  in  which  the  optimality  region 
contains  more  than  one  point. 

An  important  consequence  of  the  boundedness  of  the  optimality  region  is  that 
(see  Fiacco  and  McCormick  [8],  Theorem  24) 

(1.3)  all  level  sets  {x  £ XF  : f(x)  < t}  are  bounded. 


Once  and  for  all,  we  will  require  the  standard 

(1.4)  Constraint  qualification  of  Slater  [20]:  There  exists  a point  x £ Rn 
such  that  for  all  constraints  gfix)  > 0,  i = 1, 

gfix)  > 0. 


That  condition  guarantees  that  the  interior  intXF  of  the  feasibility  region  is 
nonempty,  and  that  none  of  the  constraint  functions  vanishes  identically  on  XF . 

The  Slater  constraint  qualification  is  clearly  necessary  and  sufficient  for  the  defi- 
nition of  the  “ logarithmic  barrier  function77 

m 

(1.5)  5Zln[$i(x)], 

1=1 


which  tends  to  -oo  as  r approaches  the  boundary  of  the  feasibility  region  XF . Use 
of  the  logarithmic  barrier  function  for  perturbing  the  objective  function  in  order  to 
approximate  optimal  solutions  has  been  suggested  early  on  by  Frisch  [9] . Following 
the  general  SUMT  approach,  a family  of  (logarithmic)  barrier  perturbations  are 
introduced  on  intXF: 


(1.6) 


L(x,r)  = f(x)  — r 


Y 


r > 0, 


These  functions  are  clearly  continuous  and  convex  in  the  variable  x for  each  spe- 
cific parameter  value  r.  They  grow  infinitely  large  close  to  the  boundary  of  the 
feasibility  region  XF . Any  minimizer 


x(r)  :=  argmin xL(x:r) 


of  such  a barrier  perturbation  is  therefore  an  unconstrained  minimizer.  The  idea  is 
to  reduce  the  perturbation  successively  in  order  for  the  unconstrained  minimizers 
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x(r)  to  converge  towards  a - constrained  - optimal  solution  x*  E X *. 

It  is  an  important  result  that  those  minimizers  always  exist  - even  if  the  feasiblity 
region  XF  is  not  bounded  - , provided  the  optimality  region  X * is  bounded 
(assumption  (1.2)).  It  then  follows  that  the  minimizers  accumulate  towards  the 
optimality  region.  Indeed  the  following  statement  (see  Fiacco  and  McCormick 
[8],  Theorem  25)  - which  extends  also  to  classes  of  barrier  functions  other  than 
the  logarithmic  ones  - is  the  basis  of  barrier-driven  Sequential  Unconstrained 
Minimization  Techniques  (SUMT)  pioneered  by  Fiacco  and  McCormick  [5],  [6], [7]. 

(1.7)  If  the  optimality  region  X*  of  the  convex  programming  problem  (1.1) 
is  nonempty  and  bounded,  then 

(i)  each  barrier  perturbation  L(x,r  ):  r > 0 has  a minimizer  x(r); 

(ii)  the  values  of  minimizers  x(r)  converge,  that  is, 

f[x(r)]  — > t*  as  r — > 0 

holds,  where  x(r)  is  any  minimizer  of  the  barrier  perturbation  L(x,r); 

(iii)  r < r implies  f[x(r)]  < f[x(r)\  for  any  minimizers  x(r)  and 
x(f),  respectively. 

Under  very  general  conditions,  the  minimizers  x(r)  will  be  unique.  Then  those 
minimizers  describe  a continuous 

(1.8)  “(primal)  trajectory ” x = x(r),  r > 0. 

Indeed,  any  sequence  of  minimizers  x(rf),ri  — > r0  > 0,  must  have  an  accumu- 
lation point  x by  (1.7, iii)  and  (1.3).  Since  L(x,r)  is  continuous  in  both  x and 
r,  L[x(ri),ri\  < L[z(r0),  r*]  implies  in  the  limit  L(x,r0)  < L[x(r0),  r0].  As  the 
minimizer  x (r0)  is  unique,  x = x(r0).  Thus  x (r*)  converges  towards  x(r0),  whence 
the  trajectory  x = x(r)  is  continuous  at  any  r — r0  > 0. 

The  trajectory  x = x(r)  and  its  relatives  have  received  considerable  attention 
McCormick  [13],  [14],  Anstreicher  [2],  Bayer  and  Lagarias  [3],  [4].  An  algorithmic 
convergence  theory  based  on  the  concept  of  “ self- concor dan f barrier  functions 
has  been  developed  by  Nesterov  and  Nemirovskii  [16],  [17].  The  same  trajectories 
generally  arise  for  the  method  of  centers  proposed  originally  by  Huard  [10], [11], 
investigated  later  by  Sonnevend  [21]  and,  for  the  case  of  linear  programming  prob- 
lems, by  Renegar  [18], [19].  The  projective  interior  point  method  proposed  and 
proved  to  be  of  polynomial  complexity  by  Karmarkar  [12]  is  based  on  related  con- 
cepts. 
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If  there  is  a unique  optimal  solution  x*,  that  is,  if  the  optimality  region  X*  con- 
sists of  a single  point,  then  the  primal  trajectory  converges  to  x*  in  view  of  (1.7). 
But  what  happens  if  there  are  multiple  optimal  points?  Does  a limit  exist,  and  if 
so,  how  can  that  particular  optimal  point  be  characterized? 

In  the  linear  programming  case,  the  feasibility  region  XF  is  a closed  convex  poly- 
hedron, and  the  optimality  region  X*  is  either  a single  vertex  or,  in  the  case  of 
multiple  optimal  solutions,  an  entire  exposed  set  or  ( “face”)  of  that  polyhedron 
such  as  an  edge  or  a higher  dimensional  analogue  (see  for  instance  Stoer  and  Witz- 
gall  [22]).  Megiddo  [15]  observed  and,  Adler  and  Monteiro  [1],  Witzgall,  Boggs, 
and  Domich  [23])  proved,  in  more  general  settings,  that  the  minimizers  x(r)  con- 
verge towards  the  “analytic  center”  of  the  optimal  face.  One  goal  of  this  work  is 
to  investigate  generalizations  of  this  result. 

We  call  a convex  programming  problem  (1.1)  “differentiable”  if  the  functions  f(x) 
and  gi(x),i  = 1 ...to,  are  differentiable  in  the  interior  of  their  definition  region  C, 
and  if  the  feasibility  region  is  contained  in  that  interior: 

XF  C intc. 

For  such  differentiable  convex  programming  problems  it  is  possible  to  define  a 
dual  trajectory  associated  with  the  primal  trajectory  x = x(r).  Indeed,  in  view  of 
the  convexity  and  differentiablility  of  the  logarithmic  barrier  perturbation  L{x,  r), 
vanishing  gradients  are  necessary  and  sufficient  to  characterize  its  minimizers  x(r). 
Considering  gradients  as  column  vectors,  we  may  write  that  optimality  condition 
as  follows: 

TTL 

(1.9)  V/[x(r) ] = V&[s(r)]  . 

i= i 9nx\r)\ 

Here  the  vectors  u(r)  with  nonnegative  components 

(1-10)  U;(r)  = -tAtt,  i = 

9t[x{r)\ 

are  usually  unique  and,  if  so,  define  a 

“ dual  trajectory ” u — u(r) 
which  shadows  the  primal  trajectory. 

This  trajectory  relates  to  the  “Wolfe-dualization”  [24]  of  the  convex  programming 
problem  (1.1): 
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(1.11)  Maximize  f{x)  — Y%Li  9i{x)ui  subject  to 


V/(z)  - 

i=i 


'LL  — (u-i , 


> 


0, 

0, 


x E C. 


The  combination  [i(r),w(r)]  constitutes  for  every  parameter  r > 0 a feasible  solu- 
tion of  the  Wolfe-dualization  (compare  [7]).  Furthermore,  consider  the  - necessary 
and  sufficient  - KKT  optimality  condition  for  an  optimal  solution  x*  of  the  primal 
convex  programming  problem.  It  requires  the  existence  of  an  m— vector  u*  > 0 of 
KKT  multipliers  such  that 

v/(x*)  = £V»(**K, 

while  satisfying  the  complementary  slackness  condition 
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Y,gi  O’K  = o. 

7 — 1 

Note  that  those  conditions  also  imply  that  (x*,u*)  is  an  optimal  solution  to  the 
Wolfe- dualized  problem  (1.11)  and,  conversely,  that  if  (x*,u*)  is  an  optimal  solu- 
tion, then  x*  is  an  optimal  solution  of  the  primal  problem  and  u*  is  a corresponding 
vector  of  KKT  multipliers. 

For  the  unique  limit  x * E X * of  the  primal  trajectory  x = x(r)  - should  it  exist 
- , we  may  consider  the  convex  set 

(1.12)  U*  = U*(x *)  = {u6  Rm  : (x*,u)  Wolfe  dual  optimal  (1.11)} 
as  a “ dual  optimality  region” . 

Indeed,  for  linear  programming  problems,  the  Wolfe-dualization  creates  the  famil- 
iar dual  linear  programming  problem  - with  the  primal  variables  dropping  out. 
The  set  U*  then  becomes  the  optimality  region  of  that  dual  linear  program,  and 
the  dual  trajectory  can  be  interpreted  as  a SUMT  trajectory  of  the  dual  linear 
programming  problem  converging  to  the  analytic  center  of  the  dual  optimality 
region.  We  will  examine  generalizations  of  this  dual  result,  too. 

The  material  will  be  presented  as  follows:  In  Section  2,  following  the  Introduction, 
analytic  centers  of  primal  optimality  regions  are  introduced,  followed  by  a section 
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of  examples  and  counterexamples.  In  Section  4,  rank-integrity  is  defined  and  key 
properties  derived.  The  first  theorem  characterizes  limits  of  the  primal  trajectory 
under  rank-integrity.  It  will  be  proved  in  Section  5.  The  last  four  sections  intro- 
duce analytic  centers  of  dual  optimality  regions,  and  prove  the  second  theorem 
about  conditions  under  which  those  analytic  centers  are  limits  of  dual  trajectories. 


2.  Analytic  Centers  of  the  Primal  Optimality  Region 

We  are  particularly  interested  in  the  case  in  which  the  optimality  region  X*  of 
the  convex  programming  problem  (1.1)  consists  of  more  than  one  point,  and  we 
wish  to  establish  a concept  of  “analytic  center  of  the  optimality  region”. 

The  classic  ([10], [11], [21])  definition  of  the  analytic  center  of  a system  of  con- 
straints 

(2.1)  gi(x)  >0,  i = l,  ...,m, 

is  as  maximizer  of  the  product  of  the  constraint  functions  or,  equivalently  if  the 
interior  intXF  of  the  feasibility  region  is  not  empty  (see  1.4),  as  maximizer  of  the 
logarithmic  barrier  function  (1.5), 

= argmaxx  ^ln[^(x)] 

\i=l 

Such  analytic  centers  exist  if  the  feasibilty  region  XF  is  bounded. 

Note  that  this  definition  pertains  to  systems  of  constraints  and  not  to  their  so- 
lution sets:  Different  systems  of  constraints  with  identical  solution  sets  may  give 
rise  to  different  analytic  centers,  each  maximizing  a different  sum  of  constraint 
logarithms.  Adding  a redundant  constraint  is  a case  in  point. 

In  the  linear  programming  case,  a system  of  constraints  which  characterizes  the 
optimality  region  can  be  obtained  in  canonical  fashion  from  the  constraints  of  the 
given  linear  programming  problem:  turn  the  inequalities  which  are  “ universally 
binding ”,  that  is,  satisfied  as  equalities  for  every  optimal  solution,  into  equations. 
The  resulting  system  of  linear  inequalities  and  equations  may  not  have  interior 
solutions.  As  a result,  the  definition  of  analytic  centers  needs  a slight  generaliza- 
tion: only  those  constraints  that  are  not  universally  binding  are  included  in  the 
sum  of  constraint  logarithms  to  be  maximized. 

In  our  more  general  case  of  convex  programming,  there  appears  to  be  no  natural 


argmax3 


^z=l 
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system  of  optimality  constraints  that  derives  from  the  feasibility  constraints,  and 
leads  to  a direct  canonical  characterization  of  the  analytic  center  of  optimality 
regions.  Furthermore,  the  optimality  region  of  programming  problems  that  are 
linear  is  always  an  exposed  set  of  the  feasibility  region.  For  nonlinear  convex 
programming  problems,  the  optimality  region  may  not  even  be  an  extreme  set. 

Returning  to  the  linear  programming  case,  the  analytic  center  of  a bounded  op- 
timality region  can  also  be  characterized  as  the  maximizer  over  the  optimality 
region  X * of  a logarithmic  barrier  function  again  constructed  from  those  feasi- 
bility constraints  which  are  not  universally  binding  at  optimality.  That  alternate 
definition  differs  conceptionally  from  the  earlier  one  in  that  the  analytic  center  is 
defined  with  respect  to  the  optimality  region  X*  as  a set  rather  than  a character- 
izing system  of  constraints.  It  is  this  characterization  of  the  analytic  center  of  the 
optimality  region  that  extends  naturally  to  convex  programming  problems.  Let 
thus 

(2.2)  Z = {1  < i < m : gt(x)  = 0 for  all  x £ X*}, 

denote  the  set  of  the  indices  of  constraints  that  are  universally  binding  at  optimal 
solutions  of  problem  (1.1). 

The  concept  of  universally  binding  constraints  is  not  as  compelling  here  as  for  lin- 
ear programming  problems.  Indeed,  any  system  of  constraints  (2.1)  can  - in  many 
ways  - be  replaced  by  a single  constraint  g(x)  > 0 with  a single  concave  func- 
tion g(x ),  and  this  single  sufficient  constraint  will  be  universally  binding,  unless 
there  are  optimal  solutions  in  the  interior  of  the  fesibility  region  XF . By  contrast, 
linear  programming  problems  with  bounded  optimality  regions  will  always  have 
constraints  that  are  not  universally  binding.  We  denote  the  index  set  associated 
with  constraints  that  are  not  universally  binding  by 

(2.3)  N = {1  < z < m : i ^ Z}, 

and  define  an  “ analytic  center  of  the  (primal)  optimality  region”  by  the  following 
optimization  problem: 

(2.4)  Maximize  JJ  gi(x)  subject  to  x £ X* . 

i£N 

If  N = 0,  then  we  consider  any  x £ X*  an  analytic  center  of  the  optimality  region. 
This  convention  may  lead  to  multiple  analytic  centers.  We  will  give  an  example 
later  where  the  above  maximization  problem  does  not  define  a unique  analytic 
center  even  if  N ^ 0.  Note  that  by  that  definition 
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(2.5)  analytic  centers  x*  form  a nonempty  convex  set  and  satisfy 


gi(x*)  > 0 for  i E N. 


Proof.  At  least  one  analytic  center  x*  exists  since  X*  is  supposed  to  be  nonempty 
and  bounded  by  assumption  (1.2). 

If  the  index  set  N is  empty,  nothing  more  needs  to  be  shown.  Otherwise,  we 
observe  that  the  optimality  region  X*  contains  a point  x with  gi(x)  > 0 for  all 
i E N.  Indeed,  let  i be  some  index  in  N.  If  gi(x)  = 0 for  every  x E X* , then 
i E Z by  definition.  Thus  there  exists  x E X*  with  gi(x^)  > 0.  Define 


where  \N\  denotes  the  cardinality  N.  Since  X*  is  convex,  x E X*  and  gi(x)  > 0 
for  all  i E N.  The  objective  function  of  problem  (2.4)  assumes  a positive  value 
for  x and,  therefore,  for  x*.  Thus  gi(x*)  > 0 for  i E N. 

This  observation  also  makes  it  possible  to  pass  to  the  logarithm  of  the  objective 
function  of  problem  (2.4),  that  is,  the  corresponding  sum  of  logarithms 

Y MftW], 

ieN 

without  affecting  the  definition  of  analytic  centers.  That  function  is  concave, 
whence  the  set  of  maximizers  convex.  □ 

We  observe,  that 

(2.6)  in  the  absence  of  constraints  that  are  universally  binding  at  optimal- 
ity, that  is,  if  Z = 0,  every  accumulation  point  of  minimizers  x(r) 
of  the  barrier  perturbation  L(x,r)  is  an  analytic  center  (2-4)  of  the 
optimality  region. 

Proof.  Because  all  constraint  indices  i are  in  N , there  exists  by  (2.5)  an  analytic 
center  x*  with  gz(x*)  > 0,2  = that  is,  x*  E intXF.  Thus  L(x*,r)  is 

defined  for  r > 0,  and 

m ttl 

L[x(r),r]  = f[x(r)]  - r ln(#[z(r)])  < f(x*)  - r]T  ln[#(z*]  = L(x*,r). 

i=i  i=\ 
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Since  f(x*)  = t* , 


0 < f[x(r)]  - t* 


< r £ ln(si[x(r)])  - ln[ffi(a:*)] 


<i= 1 


i—l 


and 

771  771 

XI  5i[a:(r-)]  > I[s;(x*). 

1=1  1=1 

Let  x G X*  be  an  accumulation  point  of  x[r)  as  r — > 0.  Then 

mm  mm 

iis.M7')]  - n ^i(^)  > iis.(z*)  - n*(*)  ^ °- 

Z=1  Z = 1 Z = 1 Z = 1 

The  left  hand  side  of  the  above  expression  converges  to  zero  by  continuity.  Thus 

771  771 

n&o)  = iis'(x*)> 

z= 1 z= 1 

which  was  to  be  shown.  □ 


The  absence  of  universally  binding  constraints  at  optimality  is,  of  course,  equiv- 
alent to  the  existence  of  optimal  solutions  in  interior  of  the  feasible  region.  Also 
if,  in  this  case,  the  analytic  center  x*  of  the  optimality  region  is  unique,  then 
(2.6)  implies  that  it  is  the  limit  of  the  primal  trajectory  x = x(r),  - which  is  the 
behavior  we  wish  to  establish  in  general.  In  the  presence  of  universally  binding 
constraints,  however,  it  may  not  hold,  as  some  of  the  examples  in  the  following 
section  illustrate. 


3.  Examples  and  Counterexamples  for  the  Primal  Limit 
Conjecture 


In  this  section  we  examine  analytic  centers  of  some  convex  programming  problems 
in  order  to  illustrate  the  issues  raised  previously.  Consider  the  linear  programming 
problem  for  x = (sq,  x2) 


(3.1) 


Minimize  — 2x2  — x3 
9i{?) 

92{x) 

93{x) 

9*(x) 

9s{x) 


subject  to 

Xi  > 0, 

x2  > 0, 
x3  > 0, 

2 — xi  — 2x2  — x3  > 0, 

4 + 3xi  — 4x2  — 2x3  > 0. 


9 


The  feasible  set  XF  is  a polytope,  and  the  optimality  region  X*  is  an  edge  of 
that  polytope,  characterized  by  g\{x)  = #4(2)  = gs(x)  = 0.  Thus  both  kinds 
of  degeneracies  are  present  at  optimality:  optimal  solutions  are  not  unique,  and 
neither  are  the  KKT  multipliers  due  to  the  linear  dependence  of  the  above  three 
constraints  gt(x),i  £ Z = {1,4,5}.  Since  this  convex  programming  problem  is 
differentiable,  the  optimality  condition  (1.9)  can  be  applied  to  characterize  primal 
trajectory  points  x = x(r).  The  gradient 


v/O)  = 


0 

-2 

-1 


can  be  expressed  in  the  form: 


1 ^ 

r 

0 1 

r 

( 0 ^ 

r 

( -1  \ 

0 

— + 

1 

— + 

0 

h 

-2 

0 } 

Xi 

0 J 

x2 

1 1 ) 

x3 

-1  / 

2 — xi  — 2x2  — x3 


4 + 3xi  — 4z2  - 2z3 


This  condition  yields  the  following  expressions  for  the  primal  trajectory: 
2 + 5r  — \/4  — 4r  + 25r2  4(v/19  + l)r 


x\ (r)  = 


\/l9  — 1 


3(2  + 5r  + y/A  - 4r  + 25r2) 


/ \ _ 2 - 5r  + V4  - 4 r + 25r2  2 - 5r  + y/i  - 4r  + 25 r2 

x2{r)  - o , x3{r)  = 


8 


Clearly, 


as  r 


0. 


x:(r)  -*  0,  x2(r ) -»  i x3(r)  -» 

These  limits  also  describe  the  solution  of  the 


1, 

optimization  problem 


(3.2) 


Maximize  x2x3  subject  to 

gi(x)  xj  = 0, 

92(2)  = x2  > 0, 

93{x)  = x3>0, 

g4(x)  = 2-Xl-2x2-x3  = 0, 

g5(x)  = 4 + 3^!  — 4z2  — 2x3  = 0, 


which  characterizes  the  analytic  center  of  the  optimality  region.  To  determine 
that  maximum,  the  reader  may  choose  to  substitute  x3  — 2 — 2x2  and  to  find  the 
unconstrained  maximum  of  2a;2(l  — x2). 
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In  trying  to  extend  this  result  to  the  general  convex  case,  however,  difficulties  are 
encountered.  Consider  the  functions 

p(zi,z2)  = i - l)2  + xj,  q(xux2)  = y(zi  + l)2  + x\, 

and 

(3.3)  f(x i,x2)  = p(x1,x2)  + q(xi,  x2)  ~ 2. 

Both  p(xi,x2)  and  q(xi,x2)  are  convex  functions  describing  circular  cones  with 
apex  at  the  points 


respectively.  The  functions  are  linear  on  each  ray  emanating  from  their  apex. 
They  are  strictly  convex  on  every  line  not  containing  the  apex.  It  follows  that 
the  function  f(xi,x2)  is  piecewise  linear  on  the  z-axis,  and  is  strictly  convex 
everywhere  else.  It  assumes  a minimum  of  zero  on  the  interval  spanned  by  the 
two  apexes. 

In  the  previous  section,  we  promised  to  provide  an  example  of  analytic  centers 
being  not  unique  even  if  there  are  nonuniversally  binding  constraints,  N ^ 0, 
and  if  the  SUMT  minimizers  x{r)  are  unique.  To  this  end  consider  the  convex 
programming  problem 

Minimize  f(x i,x2)  subject  to 

9i(x)  = x2>0 
g2(x)  = 1 - x2  > 0 

thus  has  the  bounded  optimality  region  X*  = {(cci,  x2)T  : — 1 < x1  < +1,  x2  = 0} 
with  N = {2}.  The  corresponding  function  — ln(l  — x2),  which  is  supposed  to 
characterize  the  analytic  center  of  the  optimality  region  X*,  is  constant  there. 
Thus  every  optimal  solution  is  an  analytic  center.  On  the  other  hand,  the  objective 
function  f(x i,  x2)  is  strictly  convex  above  the  x-axis.  The  resulting  barrier  pertur- 
bations L(x,r)  are,  therefore,  strictly  convex  in  the  interior  of  the  feasible  region, 
the  primal  trajectory  is  well-defined,  and  has  the  unique  limit  x*  = (0,0)T.  The 
objective  function  f(x i,z2)  is  not  differentiable  at  the  apexes  ( + 1,  0)T,  ( — 1, 0)T, 
the  focal  points  of  the  ellipses  which  describe  its  level  sets.  But  the  function 
/(xl5  x2)2  is  differentiable  and,  subject  to  the  above  constraints,  shows  the  same 
abnormal  behavior. 

The  following  examples  show  that,  even  if  a unique  analytic  center  of  the  optimal- 
ity region  exists,  it  may  not  be  the  limit  of  a well-defined  primal  trajectory.  First, 
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consider  the  convex  programming  problem  defined  on  C — Rq  = {(x\,x2)T  • 

xi,x2  > 0}. 


(3.4) 


Minimize  x2 
9i(x) 
92{x) 

9s{x) 

94{x) 


subject  to 

X\  > 0, 

2 — X\  > 0, 

x2  > 0, 

yJX\X2  > 0. 


All  barrier  perturbations  are  differentiable  in  the  interior  intXF  of  the  feasibility 
region.  So  the  optimality  condition  (1.9)  for  minimizing  a barrier  perturbation 
L(x,r)  applies: 


0 

1 


T 

*JX\X2 


For  the  point  (x1(r),  x2(r))T  to  lie  on  the  primal  trajectory  then  requires  that 


1 

Xi 


+ 


2 — X\  2x\ 


= o, 


= 1, 


It  follows  that  X\ (r)  = 6/5,  x2 (r)  = 3r/2,  and  therefore  limr_o  ^i(r)  = 6/5.  On 
the  other  hand,  N = {1,2}  and  so  the  center  (xl,xl)T  of  the  optimality  region 
is  characterized  by  x*2  — 0 and  (see  (2.4))  by  the  following  maximization  problem 
for  X\. 

Maximize  Xi{2  — x^)  subject  to  0 < X\  < 2. 

The  solution  is  clearly  x\  = 1 ^ 6/5.  The  center  of  the  optimality  region  thus 
differs  from  the  limit  of  the  primal  trajectory  for  this  example.  Note,  that  the 
concave  constraint  functions  have  vertical  tangents  at  boundary  points  of  the 
feasible  region,  that  is,  for  x1  = 0 and  x2  = 0.  They  are  therefore  not  differentiable 
at  those  points. 

An  even  stronger  counterexample  to  the  conjecture  that  a unique  analytic  center 
is  the  limit  of  the  primal  trajectory  x = x(r),  refers  back  to  the  function  f(x i,  x2 ) 
defined  above  (3.3). 


(3.5) 


Minimize 


f(xi,x 2)  subject  to 

gY(x)  = x1  > 0 
g2(x)  = x2>0. 
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The  feasibility  region  contains  interior  points,  and  the  optimality  region  X*  = 
{(xi,X2)t  : — 1 < < +1,  x2  = 0}  is  bounded.  Again,  the  barrier  perturbation, 


L(x1:x2,r)  = yj{x i - l)2  + x\  + y(xa  + l)2  + x\  - r[ln(x1)  + ln(x2)],  r > 0, 

is  differentiable  in  the  positive  orthant  R ^ =.  {(xi,x2)t  : xi,x2  > 0}.  The 
necessary  and  sufficient  optimality  condition  for  the  SUMT  minimizer 


x 


(r)  = 


thus  is 

(3.6) 

(3.7) 


X\  — 1 


+ 


xi  = Xi(r) 

x2  = x2(r) 


Xi  + 1 


\J(X i ~ l)2  + x2  \J(X i T l)2  T 

X2  , ^2 


— 7* 


s) 


+ 


r|T) 

X2J 


yj(x i - l)2  + x|  + l)2  + ^2 

For  the  following  calculations,  we  abbreviate: 

:=  ^/(x!  - l)2  + x\  = \jx\  + x\  + \ -2xy 

V+  :=  \/(zi  + l)2  + x^  = yjxl  + x%  + 1 + 2x:. 

The  optimality  condition  is  a system  of  linear  equations  for  1 / and  1 />/+  with 
solutions 


1 


r / 1 xi  + 1 
2 Vxi 


1 


x‘ 


'2  ) V+ 

or,  since  Xi  and  x2  are  both  positive, 


r ( x i — l 
2 ~~ 


x; 


Xi 


-r(x2  — x2  — Xi)\/— 


2xix2  = 

2xix2  = — r(x2  — x 2 — Xi)V-f, 


and 


(Xg  — X2  — Xi)v^  = (x?  - X*  - xOV+. 

Squaring  both  sides  of  the  above  equation  yields  the  necessary  conditition 
(xg  — x2  — Xi)2(x2  + x2  + 1 — 2xi)  = (x2  — x\  — xx)2(x2  -f  x\  + 1 + 2xi). 
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We  collect  by  common  factors  {x\  + x\  + 1)  and  2xi,  finding,  in  view  of 


(x\  — x\  — xa)2  — ( x\  — x\  — xj)2  = 4x(x2  — x\) 

(x2  - x2  - Xi)2  + (a?  - x2  - xx)2  = 2[(x2  - X2)2  + X2], 

that 


4xi(x2  — x2)(x2  + z2  -f  1)  = 2[(x2  — x2)2  + x2]2xx. 
Dividing  by  4xi  > 0,  yields 

(x2-x2)(x?  + x2  + l)  = (x2-x2)2  + x2. 


or,  expanded, 


2x2x2  — 2xl  — ~ 0- 

Dividing  by  x2  > 0,  finally  leads  to  the  following  necessary  optimality  condition 
for  the  SUMT  minimizers  in  this  example: 

2z1(r)2  - 2z2(r)2  -1=0. 

This  is  the  equation  of  the  arc  of  a hyperbola  with  asymptote  X\  = x2.  Also, 


= V2 


4x2  — 2 


Xi 


= V2 


( 4x1 


4x1  - 1J  A ' \4x\  + 1 

By  continuity,  and  because  x2(r)  — > 0 as  r — > 0, 


Xl 


Xi  (r) 
x2  (r) 


( 11  f ) 


as  r 


0. 


This  point,  however,  differs  from  the  unique  analytic  center  of  the  optimality 
region, 


x = 


Indeed,  N = { 1}  and 


J[gl{x1,x2)  = xi. 

ieN 
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the  maximum  of  which  over  the  interval  {(xi,x2)t  : — 1 < £1  < +1,  x2  = 0}  is 
clearly  assumed  for  x\  — 1. 

Again  we  note,  that  the  function  f(x i,x2)  (3.3)  is  not  differentiable  at  the  points 
( + 1,  0)T,  ( — 1,  0)T,  but  its  square,  f(x\,x  2)2  is  differentiable  at  these  points  and 
everywhere  in  XF . After  dividing  by  2f(xi,x2)  > 0,  the  optimality  condition  for 
SUMT  minimizers  then  takes  the  form  (compare  (3.6)): 


X\  — 1 X\  + 1 

\J{x  1 - l)2  -f  ^(cci  + l)2  + 

^2 ! ^2 

\/ (^i  - l)2  + A yj{x\  + l)2  + A 


r 

2 f(x1,x2) 


r 

2/(xi,x2) 


This  shows  that  the  squared  function  gives  rise  to  same  primal  trajectory  as  the 
original  function,  only  the  dependence  on  the  parameter  r is  affected.  It  follows 
that  the  discrepancy  between  trajectory  limit  and  analytic  center  may  obtain  also 
for  fully  differentiable  objective  functions  and  linear  constraints. 


4.  Rank-Integrity 

As  the  examples  in  the  previous  section  show,  the  characterization  of  the  limit  of 
the  primal  trajectory  x = x(r)  as  an  analytic  center  requires  a restriction  of  the 
class  of  programming  problems  (1.1)  to  be  considered.  We  proceed  to  define  such 
a restriction. 

We  say  that  a convex  or  concave  function  c(x)  on  Rn  has 

(4.1)  “ rank-integrity ” 

if  for  any  line  L and  any  line  segment  S = [x,x]  C L of  positive  length,  the 
following  holds: 

(4.2)  If  the  function  c{x)  is  constant  on  the  segment  S , then  it  is  constant 
on  the  entire  line  L and  on  any  line  parallel  to  L. 

We  proceed  to  characterize  functions  with  rank-integrity.  We  call  any  function  of 
a single  real  variable  6 


(4.3) 


constant-free ” 


if  there  is  no  interval  9\  < 9 < 02  on  which  the  function  is  constant.  We  then 
call  a function  (J(s),s  E Rn , “ constant-free ” if  its  1-dimensional  restrictions  are 
constant-free  (4.3). 

Constant  functions  as  well  as  convex  (concave)  constant-free  functions  - such 
as  strictly  convex  (concave)  functions  - have  rank-integrity.  In  what  follows, 
we  will  provide  more  detailed  characterizations  of  rank-integrity  without  always 
mentioning  these  trivial  cases.  We  will  also  restrict  ourselves  to  discussing  the 
convex  case  since  the  concave  case  is  analogous. 

(4.4)  A convex  function  c(x)  on  Rn  has  rank-integrity  if  and  only  if  it  is 
direct  sum  of  a constant  function  and  a constant-free  convex  function. 

In  other  words,  a convex  function  c(x)  has  rank-integrity  if  and  only 
if  it  is  of  the  form 

c(x)  = Q(Ax ), 

where  A denotes  an  l x n-matrix,  and  Q(s),  s E Rl  is  convex  and 
constant- free. 

Proof.  Suppose  the  convex  function  c(x)  has  rank-integrity  and  let  M be  the  set  of 
all  directions  w in  which  c(x)  is  constant  along  a line.  By  the  definition  of  rank- 
integrity,  those  directions  indicate  constant  behavior  regardless  of  the  starting 
point.  Thus  it  follows  immediately  that  the  set  M is  a linear  subspace  of  Rn. 
This  subspace  may  be  described  as  the  null-space  of  a matrix  A of  full  rank: 
M = {w  : Aw  = 0}.  Let  M1  denote  the  orthogonal  complement  of  M,  and  split 
any  x E Rn  into  its  two  components  in  those  orthogonal  subspaces:  x — w + x. 
Then 

c(x)  = c(w  -f-  x)  = c[x)  — c[AT (AAt)~1  Ax]  = Q(Ax). 

We  have  to  show  that  the  convex  function 

Q{s)  = c^iAAfy's] 

is  constant-free.  Assume  to  the  contrary  that  s^s  define  a constant  line  segment 
of  (?(.s).  Then  x = AT(AAT)-15  and  x — AT(AAT)~1s  are  points  in  Rn  with  s = 
Ax  and  s = Ax.  It  follows  that  the  points  x , x bracket  a constant  line  segment  S 
for  c(z).  By  rank-integrity,  c(x)  is  then  constant  everywhere  in  direction  w = x — x , 
whch  thus  must  lie  in  the  null-space  Aw  = Ax  — Ax  = s — s — 0.  This  contradicts 
the  assumption  s^fs.  The  convex  function  Q(s)  is  therefore  const  ant -free. 
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Conversely,  suppose  c(x)  = Q(Ax ) with  (J(s)  convex  and  constant-free.  Suppose 
further  that  points  x^x  are  endpoints  of  a line  segment  5 on  which  c(x)  has 
constant  values.  For  0 < 6 < 1, 

c[xA  9(x  — x)\  = Q[Ax  + 9(Ax  — Ax)\  — <5[s  + 0(s  — 5)], 

where  s = Ax,  s = Ax.  If  s s,  the  function  (J($)  would  be  constant  on  the  line 
segment  AS  C Rl  in  violation  of  the  assumption  that  Q(s)  is  constant-free.  Thus 
s = s.  This  implies  Aw  = 0 for  the  direction  w = x — x,  and 

c(x  + 6w)  — Q(Ax  A OAw ) = Q(Ax)  - c[x ) 

for  any  x and  9.  □ 

Linear  and  convex  quadratic  functions  c(x)  have  rank-integrity.  Indeed,  those 
functions  belong  obviously  to  a more  restricted  class  of  functions  with  rank- 
integrity  characterized  by  the  property: 

(4.5)  If  the  function  c[x)  is  linear  on  a segment  S,  then  it  is  linear  on  any 
line  L parallel  to  S . 

It  follows  again  that  each  such  function  is  the  direct  sum  of  a linear  function  and 
a strictly  convex  function,  that  is,  it  is  of  the  form 

c(x)  = a -f  aTx  + Q(Ax) 

with  A an  / x 72,-matrix  and  Q(s),.s  E Rl , a strictly  convex  function.  Note  that 
functions  of  the  form 

c(x)  = a + aTx  A Qi(Aix)  + ...  + Qk(Akx), 

with  strictly  convex  functions  Qi,...,Qk,  are  also  of  that  form  since  additivity 
holds:  Q\{A1x)  + Qi(A2x)  = Q(Ax)  with  Q = Qx  © Q2  the  - strictly  convex  - 
direct  sum  of  two  strictly  convex  functions  and 


We  will  now  examine  implications  of  rank-integrity  for  the  objective  and  constraint 
functions  of  convex  programming  problems.  For  convenience,  we  say  the  convex 
programming  problem  (1.1)  has  “ rank-integrity ” if  the  functions 

f{x ),  gi{x),i  = 1,  •••,77i, 
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have  rank-integrity  (4.1).  For  the  remainder  of  this  section,  we  will  assume  that 
this  property  holds.  We  first  note  that,  in  this  case, 

(4.6)  the  barrier  perturbations  L(x,r),r  > 0,  have  unique  minimizers  x{r). 


Proof.  Assume  L(x,r)  has  two  minimizers  x(r)  ^ x(r).  Because  of  the  convexity 
of  L(x,r),  the  two  minimizers  span  a line  segment  S on  which  L(x,r)  = f(x)  — 
r JLi  ln<7i(z),  r > 0 is  constant.  In  general,  if  the  sum  of  convex  functions  is  linear 
and,  in  particular,  constant,  then  each  individual  function  must  be  linear.  Thus 
In gi(x)  will  be  linear  on  line  segment  5. 

The  following  general  property  of  the  logarithm  function  is  readily  established  by 
comparing  derivatives. 

(4.7)  If  the  logarithm  of  a positive  function  7 (6)  is  linear  in  6 , then  it  is 
constant  in  6. 

The  linearity  of  \ngi(x)  on  line  segment  S thus  implies  that  gi(x)  is  constant  on  S. 
By  rank-integrity,  every  constraint  function  must  then  be  constant  and  positive 
on  the  entire  line  L D S.  That  line  is  therefore  contained  in  the  feasiblity  region 
XF. 

It  follows  that  also  the  objective  function  f(x)  = L(x,r)  Pr^ln  gi(x)  is  constant 
on  line  segment  5,  whence  on  the  entire  line  L,  again  by  rank-integrity.  Thus  f(x) 
would  have  an  unbounded  level  set  in  XF , contradicting  (1.3).  (If  the  objective 
function  f(x)  belongs  to  the  special  class  (4.5)  of  functions  with  rank-integrity 
such  as  linear  and  convex  quadratic  functions,  then  the  barrier  perturbations 
L(x,r)  are  actually  strictly  convex  in  x.)  □ 

Recall  that  there  need  not  be  any  constraints  gz(x)  which  are  not  universally 
binding  at  the  optimality  region;  in  other  words,  the  index  set  N,  on  which  the 
definition  (2.4)  of  an  analytic  center  of  the  optimality  region  is  based,  may  be 
empty.  In  this  case,  all  optimal  solutions  were  defined  to  be  analytic  centers. 
Furthermore,  as  an  example  in  Section  2 shows,  the  analytic  center  may  be  non- 
unique even  if  N ^ 0 and  the  barrier  perturbations  L(x,r)  are  strictly  convex. 
The  following  proposition  shows  that  these  difficulties  are  avoided  by  the  rank- 
integrity  assumption. 

(4.8)  The  optimality  region  X*  has  a unique  analytic  center  x*  as  defined 
by  the  maximization  problem  (2.4).  Moreover  x*  lies  in  the  relative 
interior  relintX*  of  the  optimality  region,  and  gi(x*)  > 0 for  i E N. 
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Proof.  The  existence  of  at  least  one  analytic  center  was  shown  earlier  (2.5). 

The  proposition  is  obviously  true  if  X*  consists  of  a single  point  (even  if  N = 0 as 
the  relative  interior  of  a single  point  consists  of  that  point).  We  assume  therefore 
that  X*  has  multiple  points. 

Next  we  show  that  X * is  essentially  bounded  by  the  constraints  gi(x)  with  i E N. 
Using  the  notation  “affine”  to  indicate  the  affine  hull  or  span  of  a set,  and  selecting 
a subset  of  the  constraints  by  affixing  the  subscript  X,  we  claim: 

(4.9)  X*  = {z  E affineX*  : gN(x)  > 0}  (X*  = affineX*  if  N = 0). 

The  inclusion  X*  C {z  E affineX*  : gN^x)  > 0}  is  trivial.  If  X*  = affineX*  then 
the  converse  inclusion  is  also  trivial.  In  order  to  prove  that  converse  inclusion  in 
general,  select  a point  z E relintX*,  and  let  x # E affineX*  with  x*  $ X*.  We 
have  to  show,  that  x * violates  some  constraint:  gl(x#)  <0,  i E N . 

Recall  the  definition  (2.2)  of  the  index  set  Z as  characterizing  the  constraints  in 
gz{x)  which  are  universally  binding  at  optimality,  that  is,  vanish  for  x E X*.  Note 
that  since  X*  is  convex,  there  exists  0 < 9 < 1 such  that  z + 0(z#  — z)  E X* 
for  0 < 6 < 6 (see  for  instance  Stoer  and  Witzgall  [22],  Lemma  3.2.9).  For  any 
j E Z,  the  function  g3[x  + 0(z#  — z)]  is  a constant,  namely  vanishing,  function  of 
6 on  the  interval  [0,0].  By  rank-integrity,  that  function  vanishes  for  all  values  9. 
Thus  gz{x*)  — 0. 

By  the  same  token,  the  objective  function  f(x)  is  seen  to  be  constant  at  its 
optimal  value,  whence  f(x*)  = t*.  Thus  only  the  violation  of  some  constraint, 
gi(x*)  < 0,  prevents  x * from  being  an  optimal  solution.  Since  gt(x)  does  not 
vanish  on  affineX*,  the  index  i is  not  in  Z , and  belongs  therefore  to  its  complement 
N.  This  proves  (4.9). 

The  following  characterization  of  the  relative  interior  of  the  optimality  region  is 
simply  a restatement  of  (4.9): 

relintX*  = {x  E afTX*  : gN^x)  > 0}  ^ 0. 

It  follows  again  from  (2.5)  that  any  analytic  center  is  contained  in  relintX*. 

We  now  proceed  in  analogy  to  the  proof  of  (4.6),  showing  that  the  concave  function 
L*(x)  = YheN  ln&(z)  has  a unique  maximum  on  relintX*.  Assume  to  the  contrary 
that  there  are  two  different  maxima,  that  is,  two  different  analytic  centers  x*  ^ x* . 
Then  these  points  define  a line  segment  S of  positive  length  on  which  L*(x)  is 
constant.  The  individual  summands  lng2(:r),z  E X,  must  therefore  be  linear,  and 
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(4.7)  implies  that  they  are  actually  constant  on  the  segment  S.  By  rank-integrity, 
this  constant  behavior  extends  to  the  entire  line  L D 5.  In  view  of  (4.9),  this  would 
imply  L C X* , violating  the  assumption  that  the  optimality  region  is  bounded. 
□ 


5.  Limits  of  Primal  SUMT  Trajectories 

Trajectory  points  x = x(r ) may  be  characterized  as  optimal  solutions  to  an  aux- 
iliary programming  problem, 

(5.1)  Problem  Br  : Maximize  X)i€jvln[<7i(cc)]  subject  to 

f(x)  < f[x{r)} 
gi(x ) > 0 for  i £ N 
gi(x)  > gi[x(r)\  for  i £ Z. 


Indeed,  we  will  show  that 

(5.2)  for  any  r > 0,  x(r)  is  the  unique  optimal  solution  of  the  maximization 
problem  Br. 

Proof.  Let  x be  an  arbitrary  feasible  solution  of  problem  Br  above.  Note  first  that, 
by  the  monotonicity  of  the  logarithm  function  and  the  above  conditions  on  cc, 

ln(5i[x(r)])  - ^ln[Si(z)] 


ieN 


ieN 

m 


= X^Ks.^O)])  - x;in[s,(z)]  - 5Z(ln(5i[x(r)])  - Ms;(z)]) 

i=l  1=1  i£Z 

m m 

Z = 1 Z = 1 

Next,  because  of  the  constraint  f(x)  < f[x(r)j  and  since  r > 0, 

m m 

’•SMs.K7’)])  - rSMs.O)] 


2 — 1 


2 = 1 


= fix)  -rIZln[s-(x)]  ~ flx(r)]  -r2Zln(ft[a:(r)I)  + f[xir)}  - f(x) 


i=l 


i=l 


> L{x,r)  — L[x(r),r)  > 0. 
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This  shows  that,  in  order  to  maximize  problem  Br,  it  is  necessary  for  x to  minimize 
the  barrier  perturbation  L(x,r ).  Proposition  5.2)  follows,  as  x(r)  is  the  only 
minimizer  by  (4.6)  and  also  clearly  a feasible  solution  of  problem  Br.  □ 

We  are  now  ready  to  prove  the  first  of  two  theorems. 

(5.3)  Theorem  1:  If  the  optimality  region  of  a convex  programming  prob- 
lem (1.1)  is  bounded,  and  if  the  Slater  constraint  qualification  (1.4) 
holds,  then  the  limit  of  the  primal  trajectory  x — x(r ) (1.8)  for  r — » 0 
is  characterized  as  the  unique  analytic  center  x*  (2.4)  of  the  optimal- 
ity region  X* , if  (i) the  functions  f(x),  gi[x),i  = l...m,  in  the  convex 
programming  problem  have  rank-integrity  (4.1);  or  if  {j\)the  optimality 
region  has  interior  points  and  the  analytic  center  is  unique. 

Proof.  The  second  part  of  the  theorem  is  a corollary  of  the  proposition  (2.6). 

As  to  the  first  part,  there  exists  by  (4.8)  a unique  analytic  center  x*  £ X* , 
and  Piv(^*)  > 0 holds.  Consider  a sequence  { ry },  j = 1,...,  of  positive  barrier 
parameters  decreasing  to  0.  According  to  the  basic  fact  (1.7),  requiring  only 
continuity  assumptions,  the  sequence  of  minimizers  x[rj)  has  an  accumulation 
point  x with 


f[x(rj)}  t*  = f(x),  x £ X *,  gz(x)  = 0. 

Without  loss  of  generality,  we  may  assume  that  the  accumulation  point  is  actually 
the  limit 


x(rf)  — » x as  j — > oo, 

of  that  sequence,  as  otherwise  we  would  select  a suitable  subsequence.  We  now 
define 

(5.4)  x(rj)  — x(rj)  -f  x*  — x, 

where  x*  is  the  unique  analytic  center.  The  idea  of  the  proof  is  to  utilize  (5.2)  by 
establishing  x(rj),  for  sufficiently  large  j , as  a feasible  solution  of  problem  Brj, 
thus  gaining  an  upper  bound  on  corresponding  values  of  its  objective  function. 

First,  note  that  x(rj)  — > x*  follows  by  definition  (5.4).  In  view  of  g^(x*)  > 0, 
gN^i^j)]  > 0 will  hold  for  suitably  large  indices  j . 

Next,  we  observe  that  the  functions  f(x),  gt(x),i  £ Z , are  constant  on  the  line 
segment  S = {x  + 0(x*  — x)  : 0 < 9 < 1}.  Indeed,  the  ends  of  S are  in  the  convex 
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optimality  region  X *,  whence  /(x)  is  constant  on  S,  and  gz{%)  vanishes  there. 
By  rank-integrity,  the  above  functions  are  constant  everywhere  in  the  direction 
x*  — x.  Since  x(r)  — x(rj)  = x*  — x by  definition,  it  follows  that  there  is  constant 
behavior  in  that  direction,  too: 

(5.5)  f[x(ri)}  = f[x{rj)},  9z{i(r,))  = sz[x(r\,)]. 


These  relations,  together  with  the  previous  observation,  ensure  that  x(r3)  is  a 
feasible  solution  for  problem  Bri.  Since  x(rj ) is  the  optimal  solution  of  that 
problem, 

£ In ?;[*(»•;)]  < ^ lnSl[i(r3)], 

zeN  ieN 


or,  in  terms  of  products  of  constraints, 


n 9i[5(ri)] 

ieN 


< n siK’j)]- 

ieN 


In  the  limit  j — > oo,  the  above  relation  becomes 


n »(*•)  ^ iiff*(5)' 

ieN  ieN 


Thus  x is  a maximizer  of  problem  (2.4).  By  (4.8),  x = x* . Hence  every  accumu- 
lation point  of  the  trajectory  x = x(r)  coincides  with  x*,  proving  Theorem  1.  □ 


6.  Analytic  Centers  of  the  Dual  Optimality  Region 

For  differentiable  convex  programming  problems,  we  will  also  investigate  limits 
of  the  dual  trajectory  (1.10)  defined  in  the  Introduction,  and  again  we  wish  to 
generalize  results  known  to  hold  for  linear  programming  problems. 

In  what  follows,  we  will  always  assume  that 

(6.1)  the  primal  trajectory  x = x(r),  r > 0 has  a unique  limit  x*  as  r — » 0. 

Note  that  the  limit  x*  is  not  required  to  be  the  analytic  center  of  the  primal 
optimality  region.  We  then  revisit  the  set  (1.12) 

U*  = {ueRm:  (x*,  u)  Wolfe  dual  optimal  (1.11)} 
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which  plays  the  role  of  a dual  optimality  region.  U*  is  convex.  To  see  this,  recall 
that  by  definition  of  the  Wolfe  dualization,  the  vectors  u E U*  are  the  KKT 
multipliers  confirming  the  primal  optimal  solution  x*.  It  is  readily  seen  that  if  u1 
and  u2  are  both  vectors  of  KKT  multipliers  for  the  same  primal  optimal  solution 
x*,  then  so  is  any  convex  combination  Xu]  -f  pu2,  A + /z  = l,A,/z>0. 

In  order  to  dualize  the  concept  of  analytic  center,  we  introduce  the  index  sets 

(6.2)  V = {1  < i < m : ut  = 0 for  all  u E U*} 
and  its  complement 

(6.3)  M = {l<z<m:  i V}, 
where  m denotes  the  i— th  component  of  vector  u. 

In  view  of  the  complementary  slackness  condition  gi(x*)ut  = 0 for  KKT  multipli- 
ers, we  can  express 

(6.4)  U*  = {u  > 0 : V/(x*)  — ^ X7gi(x*)v,i  = 0,  ut  — 0 for  i E V}. 

ieM 

We  now  define  the  “ analytic  center  of  the  dual  optimality  region ” by  the  following 
optimization  problem: 

(6.5)  Maximize  JJ  U{  subject  to  u E U*. 

ieM 

For  M = 0,  we  admit  every  point  u*  E U*  as  an  analytic  center.  Again  it  will 
be  convenient  to  reformulate  this  definition  in  terms  of  logarithms.  To  this  end 
we  have  to  ensure  that  the  dual  optimality  region  contains  points  for  which  those 
logarithms  are  defined. 

(6.6)  There  exists  a point  u E U*  such  that  ul  > 0 for  all  i E M.  Conse- 
quently 


i^M  =>  — 0. 


Proof.  If  M = 0,  the  above  is  trivial.  Let  then  j be  some  index  in  M . If  Uj  = 0 
for  every  u E U* , then  j would  by  definition  be  in  V.  Thus  there  exists  a point 
uW  E U*  with  u"P  > 0.  Define 


u — 
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where  \M\  is  the  cardinality  of  M.  Since  U*  is  convex,  u G U*  and  Ui  > 0 for  all 
i G M . Also,  the  complementary  slackness  condition  inherent  in  the  definition  of 
U*  holds  and  implies  gi(x*)ui  = 0 for  all  i.  Since  Ui  > 0 for  i G M,  the  second 
part  of  the  proposition  follows.  (Note  that  if  x*  is  a “strict  minimizer” , that  is, 
gt(x*)  = 0 if  and  only  if  i £ Z,  then  (6.6)  implies  M C Z).  □ 

Because  of  (6.6),  problem  (6.5)  can  be  reformulated  as 
(6.7)  Maximize  ui  subject  to 

Ui  > 0 for  i G M and  Ui  = 0 for  i G V. 


Indeed,  the  objective  function  of  problem  (6.5)  has  a positive  value  for  any  u as 
defined  above.  Hence  its  maximizer  u*,  - if  it  exists  -,  saisfies  u*  > 0 for  all 
i G M.  The  constraints  uz  > 0 in  (6.5)  may  therefore  be  replaced  by  Ui  > 0 
without  affecting  the  solution  of  the  problem.  The  same  is  true  if  the  objective 
function  is  replaced  by  its  logarithm. 

Note  that,  since  the  sum  of  the  logarithms  lnu^  is  a strictly  concave  function  of 
the  variables  Ui:i  G M,  and  since  the  remaining  variables  are  fixed  at  zero, 

(6.8)  if  an  analytic  center  u*  of  the  dual  optimality  region  U*  exists,  then 
it  is  unique. 

Again  we  conjecture  that,  for  convex  programming  problems,  dual  analytic  centers 
characterize  the  limits  of  dual  trajectories  in  much  the  same  way  as  primal  analytic 
centers  do  for  primal  trajectories,  and  we  are  interested  in  finding  conditions  under 
which  that  conjecture  can  be  decided. 


7.  Examples  and  Counterexamples  for  the  Dual  Limit  Con- 
jecture 

Returning  to  our  first  example  (3.1),  we  recall  that  for  the  limit  of  the  primal 
trajectory  x = x[r), 

f 

x(0)  = 
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0 \ 
1/2 

1 / 


and  that,  for  r > 0 


X\ (r)  3 + y/l9  Xi  (r)  —2  -f  \/l9 

2 — xx(r)  — 2x2(r)  — x3(r)  5 5 4 -fi  3zi(r)  — 4x2(r)  — 2x3(r)  15 

These  expressions  define  the  dual  trajectory  u = u(r)  and  yield,  in  view  of  the 
primal  trajectory  limit  x(0),  the  following  dual  limits  as  the  barrier  parameter  r 
tends  to  zero: 


ui(r) 

u2{r) 

w3(r*) 

uA(r) 

Us{r) 


xi(r) 

r 

x2  (r) 
r 

a^3  (r) 


ui(0)  = 


— 1 + y/l9 


u2(0)  = 0, 

w3(0)  = 0, 


2 - Xi(r)  - 2 x2(r)  - z3(r) 
r 

4 + 3xi(r)  - 4z2(r)  - 2z3(r) 


u4(0)  = 


8 + 719 


ii5(0)  = 


15 

7 — \^19 
30 


Because  the  primal  programming  problem  is  a linear  programming  problem,  Wolfe 
dualization  (1-11)  produces,  independently  of  x,  the  familiar  dual  linear  program- 
ming problem: 


Maximize  — 2ti4  — 4it5 

subject  to 

iii  — u4  + 3 u5 

= o, 

u2  — 2 u4  — 4 u$ 

= -2, 

u3  — u4  — 2u5 

= -1, 

u > 0. 

The  optimality  region  of  this  linear  programming  problem  is  an  edge  characterized 
by  the  additional  constraints  u2  = u3  = 0.  Thus  V = {2,3},  and  since  the 
equations  in  the  above  problem  are  equivalent  to  the  gradient  expression 


V/(x*) 


° j 

/ 1 ^ 

( ~i  \ 

3 \ 

-2  = 

0 

Ui  + 1 —2 

U4  + 

-4 

-1  / 

V o } 

V -1  ) 

-2  / 

Ui,  u4,  u5  > 0, 


it  is  verified  that  the  optimal  region  in  question  is  indeed  U*  as  defined  in  (6.4). 
The  analytic  center  u*  of  that  dual  optimality  region  is  then  defined  by  the  fol- 
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lowing  maximization  program  (compare  (6.5)) 


(7.2) 


Maximize  uiu4u$ 

subject  to 

U\  — u4  -f  3 u5 

= 0, 

u2  — 2 u4  — 4ti5 

= -2, 

U3  — U4  — 2li5 

= -1, 

u2  = u3 

= 0, 

which  can  be  treated  as  an  unconstrained  problem.  For  instance,  expressing  u3 
and  U4  by  u5,  the  reader  finds  U1U4U5  = lOug  — 7 u\  + u$.  Differentiating  yields  a 
quadratic  equation  solved  by  = (7  — \/l9)/30  since  Ui  = 1 — bu5  > 0 requires 
U5  < 1/5.  Thus  u(0)  = u*  follows. 


The  limits  u*  admit  several  interpretations.  First,  they  can  serve  as  KKT  multipli- 
ers in  the  optimality  criterion  for  the  given  programming  problem  (3.1).  Second, 
the  subset  u*,i  € Z = {1,4,5},  of  that  full  complement  of  KKT  multipliers  fur- 
nishes KKT  multipliers  for  the  optimization  problem  (3.2). 


For  general  convex  programming  problems,  there  are  again  exceptions.  Consider 
the  problem 


(7.3) 


Minimize  x2 

subject  to 

9 l(z) 

— Xx  > 0, 

9t(x) 

= X2  > 0, 

93(2) 

= x2  — x\  > 0 

The  optimality  condition  for  the  minimizer  x = x(r)  of  a barrier  perturbation 
L(x,r)  takes  the  form 


The  primal  trajectory  is  therefore  characterized  by  the  equations 


1 2cci 

Xi  x2  - x\ 


r r 

0,  I 

x2  x2  - 


x\ 


1 . 


Note  that  because  of  x2{r)  = 3xi (r)1 2  the  primal  trajectory  is  tangential  to  the 
constraint  plane  x2  = 0 at  the  minimizer  x*  = (0,  0).  It  is  easy  to  express  the 
trajectory  coordinates  in  terms  of  r > 0: 


T = 


6xl 

~5~ 


x\(t ) = 


x2(r) 


5 T 

y* 
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As  r tends  to  zero,  we  thus  find  the  limits 


ui(0)  = 0, 


“2(0)  5’  x2{r)  - Xl{rY  “3(0) 


Xi  (r)  2:2(7’) 

On  the  other  hand,  M = {2,  3},  V — {1},  and  by  (6.4) 


U*  = {u  : u2  + u3  = 1,  U\  — 0}. 

Thus  the  optimization  problem  that  characterizes  the  dual  analytic  center  (6.5) 
becomes 


Maximize  u2u3  subject  to 

u2  + u3  = 1,  u2,u3  > 0. 

The  unique  solution  to  this  problem  is,  however,  u *2  — 1/2  ^ ^2(0)  and  u\  = 
1/2  ± u3( 0). 

8.  Characterizing  Dual  Trajectories 

Portions  of  the  dual  trajectory  u = u[r)  can  again  be  characterized  as  optimal 
solutions  of  a family  of  auxiliary  problems  to  be  formulated  below. 

We  will  use  the  notation 

Vg(x)  = [Vsi(z),  ...  , Vgm(x)] 

to  indicate  the  n x m-matrix  whose  columns  are  the  gradients  of  the  constraint 
functions.  Index  sets  are  again  used  in  a self  explanatory  fashion  for  extracting 
subvectors,  such  as 


gz{x*),  uM{r): 


or  submatrices  such  as 


Vys(x). 

Index  sets  are  sometimes  used  as  subscripts  just  to  indicate  the  index  range  of 
a vector  as,  for  example,  in  the  definition  (8.1)  below  of  the  vector  dc(r).  Also, 
reciprocals  and  logarithms  of  vectors  such  as  in 

1 /u(r):  lnw(r) 
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are  to  be  understood  as  vectors  of  reciprocals  and  logarithms,  respectively,  of 
components. 

Let  again  x*  denote  the  limit  of  the  primal  trajectory  x = cc(r),  which  is  assumed 
to  exist  (6.1),  and  recall  the  index  set  M and  its  complement  V (6.2).  Split  index 
set  M into  two  portions  B and  (7,  B U C = M,  B fl  C = 0,  such  that  VgB(x*)  is 
a basis  of  the  matrix 

Let  the  superscript  indicate  formation  of  the  Penrose-Moore  generalized  in- 
verse of  a matrix.  Because  VgB(x*)  has  full  column  rank, 

V<7B(s*)+  = [VSs(x*)rV5B(x*)]-1VSB(a:*)T. 

Since  VgB(x*)  is  the  limit  of  V^s[cc(r)]  as  r — > 0,  the  latter  has  full  rank  for 
sufficiently  small  values  of  r > 0.  For  such  values  of  r we  define  the  vector 

(8.1)  dc(r ) = — 1-t  - V?c[x(r)]TV3B[i(r)]+r^— , 

uc{r)  uB[r) 

which  will  set  up  a correction  term  for  the  problem  below,  where  u(r)  is  the  dual 
trajectory  as  defined  by  (1.10)  in  the  Introduction. 

(8.2)  Problem  Qr  : Maximize  In ub  H-lnuc  — dc(r)Tuc  subject  to 

[uB,uc]  > 0, 

V/[cc(r)]  - VgB[x(r)]uB  ~ Vgc[x(r)]uc  - V gv[x(r)}uv(r)  = 0, 
where  the  index  sets  B , C partition  the  index  set  M . 


(8.3)  The  vector  \ub(t)^uc{t)]  solves,  for  all  sufficiently  small  r > 0,  the 
convex  maximization  problem  Qr. 

Proof.  By  its  definition,  the  concatenated  vector  [us(r), uc(r)\  is  positive  and 
satisfies  the  equation  constraints  of  problem  Qr.  It  is  therefore  feasible.  For  it  to 
be  also  optimal,  it  suffices  that  there  exist  a vector  of  Lagrange  multipliers  w(r) 
such  that 


-1  /uB(r) 

\7gB[x(r)]T 

— l/uc(r)  -f  dc(r) 

_ Vgc[x{r)}T  _ 

(8.4) 

Such  a vector  w(r)  indeed  exists: 

w(r)  = -X7gB[x{r)]+T 


w 


(r). 


uB(r)' 


□ 
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Several  other  important  facts  about  the  dual  trajectory  u — u(r)  can  be  ascer- 
tained under  general  conditions  following  McCormick  [14]. 

(8.5)  the  dual  trajectory  u = u(r)  remains  bounded,  as  r > 0 tends  to  zero. 


Proof.  Let  x*  be  the  limit  of  the  primal  trajectory  x(r)  with  respect  to  which  the 
dual  trajectory  is  defined.  Assume  that  u(r)  is  not  bounded.  Then  there  exists  a 
sequence  Tj  > 0,  j = 1, ...  , which  converges  to  zero  such  that  ||it(r,J-)||  — » oo.  The 
sequence  of  unit  vectors 

IKr;)li 


has  a nonzero  accumulation  point  u > 0,  ||u||  = 1,  which  we  may  assume  to  be 
a limit.  (Otherwise  we  would  select  a suitable  subsequence.)  Passing  to  the  limit 
of  the  relation 




II“0j)II 


V/[x(r3)] 


Yi  V0i[®(»v)] 
2 = 1 


ui(ri) 

iK^)ir 


we  find  by  continuity 

m 

0 = ^Vgi{x*)vi. 

i= 1 

At  least  one  component  Vh  is  positive.  Thus 


-V5a(x*)  = 

i*h  V h 


The  ratios  Vi/vh  > 0 can  be  interpreted  as  KKT  multipliers  for  the  problem  of 
maximizing  the  function  gh{x)  subject  to  the  remaining  constraints  gz(x)  > 0,  i ^ 
h.  In  addition,  complementary  slackness  holds  since  gt(x*)  > 0 implies  for  r — > 0 


u 


.(r-) 


5,[a:(r)] 


and  therefore  vl  = 0.  We  also  note  that  for  the  same  reason  gh{x*)  = 0.  Since 
all  functions  involved  are  concave,  the  KKT  conditions  suffice  to  ascertain  that 
gh{%*)  = 0 is  the  maximum  value  of  gh(p)  > 0 subject  to  the  remaining  constraints 
and,  particularly,  in  the  feasible  region  XF . This  contradicts  the  Slater  constraint 
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qualification  (1.4).  □ 

Proposition  (8.5)  assures  the  existence  of  accumulation  points  u of  u(r)  as  r > 0 
tends  to  zero.  Moreover,  convergence  holds  if  all  these  accumulation  points  can 
be  shown  to  be  equal.  The  next  proposition  is  a step  in  this  direction. 

(8.6)  If  u is  an  accumulation  point  of  u(r)  as  r > 0 tends  to  zero,  then 

Ui  = 0 for  i G V, 
where  x*  is  the  limit  of  x{r). 

Proof.  In  the  previous  proof,  gi(x*)  > 0 was  seen  to  imply  Ui(r)  — > 0.  Hence 
Hi  = 0 if  gi(x*)  > 0,  again  establishing  complementary  slackness.  Thus  (x*,u)  is 
an  optimal  solution  to  the  Wolfe  dualization  (1.11).  The  proposition  then  follows 
by  the  definition  (6.2)  of  the  index  set  V.  □ 

If  the  vectors  dc{r)  and  uy(r)  were  to  converge  to  zero  as  x(r)  converges  to 
x* , then  the  maximization  problems  Qr  (8.2)  would  tend  to  the  maximization 
problem  (6.5),  which  defines  the  analytic  center  of  the  dual  optimality  region. 
This  observation  defines  our  approach  to  proving  this  analytic  center  to  be  the 
limit  of  the  dual  trajectory.  We  also  note,  that 

(8.7)  the  analytic  center  of  the  dual  optimality  region  exists  and  is  unique. 

Proof.  For  existence,  it  has  to  be  shown  that  the  feasible  region  of  the  the  maxi- 
mization problem  (6.5)  is  bounded,  which  follows  by  an  argument  analogous  to  the 
proof  of  proposition  (8.5).  Uniqueness  follows  from  the  fact  that  the  logarithmic 
objective  function  is  concave  as  stated  earlier  in  (6.8).  □ 


9.  Limits  of  Dual  Trajectories 


The  characterization  of  the  dual  trajectory  limit  as  an  analytic  center  requires 
additional  assumptions  such  as  the  following. 
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(9.1)  (i)  The  constraint  functions  gi(x ),  i = 1 , ...,m,  are  linear. 

(ii)  If  x*  is  the  'primal  trajectory  limit,  then 


x(r)  — x* 
r 


< constant , 


that  is,  the  above  vector  remains  bounded  as  r > 0 tends  to  zero. 

Note  that  example  (7.3)  violates  those  assumptions.  We  are  now  able  to  state  our 
second  theorem. 


(9.2)  Theorem  2:  If  the  primal  trajectory  x = x(r ) converges  to  a unique 
optimal  solution  x*  E X* , and  if  one  of  the  assumptions  (9.1.i,ii)  holds 
then  the  dual  trajectory  u — u(r)  converges  to  the  analytic  center  u* 
of  the  dual  optimality  region  U* . 

The  proof  of  this  theorem  will  be  based  on  the  properties  of  the  auxiliary  maximiza- 
tion problem  Qr  (8.2),  which  by  (8.3)  characterizes  trajectory  points  u = u(r ). 
Indeed,  we  will  show  that 


(9.3)  Theorem  2 holds  if 
(i)  the  vector 

dc(r)  = — - Xgc[x(r)]TVgB[x(r)]+T-^—, 

ucyr ) ub[t) 

in  problem  Qr  vanishes  as  r > 0 tends  to  zero ; 

and  (ii)  for  each  accumulation  point  u of  the  dual  trajectory  u(r),  the 
components  U{  with  i E M are  positive. 

Proof.  We  first  note  that  uy  converges  to  zero  by  (8.6).  Therefore  we  need  to 
concern  ourselves  only  with 

UM(r)  = [uB{r\  uc{r )]. 

By  (8.5),  the  trajectory  u(r)  possesses  an  accumulation  point  u,  whence  there 
exists  a sequence  rj  > 0,  j = 1, ...  , such  that 

u(rj ) — > u for  j — > oo. 

By  its  definition  u(r0)  = [ujb(7’j),  uc(rj ),  uv{rj)\  satisfies 

V f[x{rj)\  ~ TJ gM[x(r jj\uM(r j)  - X gvlxfrffuvirj)  = 0. 
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By  the  continuity  of  the  gradients  and  again  by  (8.3),  we  find  in  the  limit  ry  — » 0 
that 


V/(z*)  - VgM{x*)uM  = 0. 


Thus  um  is  a feasible  point  of  problem  (6.5)  and  therefore,  a candidate  for  analytic 
center.  We  proceed  to  show  that  um  is  an  optimal  point.  This  requires  that  there 
exist  a vector  of  Lagrange  multipliers  w*  E JRT  such  that 


(9.4)  - -z—  = V# >M(x*)Tw*. 

um 

Note  that  by  (8.3)  [u#(r),  uc(r)\  = um(x)  is  the  maximizer  for  problem  Qr:  its 
optimality  condition  (8.4)  is  satisfied  because  for  the  vector  of  Lagrange  multipliers 

w(r)  = -V  gB[x(r)]+T 

uc[r) 

the  relation 


-1  /uB{r) 

VgB[x(r)]T 

— l/uc(r)  + dc(r ) 

_ Vgc[x(r)]T  _ 

holds.  By  hypotheses,  w(rj ) has  the  limit 


uc 

and  by  continuity  um  — (^b,  uc)  must  satisfy  (9.4).  Thus  um  is  an  analytic  center 
of  the  dual  optimality  region  U*.  Since  this  holds  for  every  accumulation  point, 
and  since  the  analytic  center  of  U*  is  unique  by  (8.7),  the  proposition  follows.  □ 

We  will  now  proceed  with  the 

Proof  of  Theorem  2:  Suppose  (9.1.i)  holds.  Then,  denoting  by  g[x)  the  m vector 
of  constraints  gi[x ), 

g(x)  = Atx  - 5,  Vg(x)  = A , 

for  some  m vector  b and  some  n x m matrix  A.  In  what  follows,  index  sets  used 
as  subscripts  again  denote  corresponding  subvectors  and  submatrices. 

In  particular,  the  submatrix  AB  has  full  rank  so  that  A^AB  = I for  its  Moore- 
Penrose  pseudo  inverse  Aq  = (A^Ag)-1  A^.  By  the  choice  of  submatrix  AB , the 
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columns  of  submatrix  Ac  are  linear  combinations  of  the  columns  of  AB:  Ac  = 
AbAqAc-  By  (6.6)  gM {x*)  = 0.  Thus  Aj^x*  = bM  and,  in  particular, 


(9.5)  bc  = ATcx'  = AtcA+btAtbx * = ATcA%TbB. 

Now 

dc(r)  = — b-  - V5c[a:(r-)]TV5B[r)]+T— — 

uc[r)  uB(r) 

= ^ {gc[x(r)\  - V gc[x{r)]TX7 gB[x(r)]+T gB[x(r)]} 

= ~ (l^cx(r)  ~ bc ] - AcA%T[A%x(r)  - bBfj 

= -i( bc-ATcA%TbB ) 

= 0 


for  all  r > 0 by  (9.5).  This  establishes  condition  (9.3.i).  To  verify  condition 
(9.3-ii),  choose  an  accumulation  point  u and  a sequence  rj  > 0,  j = 1,...,  which 
converges  to  zero  such  that  the  resulting  sequence  u(rj)  converges  to  u.  The  limit 
x*  of  the  primal  trajectory  x{r)  is  an  optimal  solution  x*  E X*.  Hence  there 
exists,  by  (6.6),  a point  il  E U*  such  that  um  > 0 and  uy  = 0 hold  in  conjunction 
with  gM^x*)  = 0.  Since 

V/(z*)  - Xg(x*)u  = V/(z*)  - XgMuM  = V/(z*)  - AMuM  = 0,  uM  > 0, 

the  point  um  is  feasible  for  problem  (6.5)  and  is  thus  a potential  analytic  center. 
Define 


wm(tj)  = um(j’j)  + um  ~ um- 

Since  V/(z*)  = V gM(x*)TUM  — AJ^um  holds  for  both  um  and  the  vector 
um  — um{tj)  satisfies 

^flx(rj)]  ~ V gM[x{rj)]uM  ~ V gv[x(rj)]uv(rj) 

= V/[x(rj)]  - Amum  ~ Avuv(tj)  = 0. 

um{t° j)  converges  to  um  > 0.  Hence  we  may  assume  UM{rj)  > 0 (or  otherwise 
select  a suitable  subsequence).  um{x0)  is  therefore  a feasible  solution  to  problem 
QTj . If  the  limit  of  any  component  of  um^tj)  equals  zero,  the  objective  function 
values  of  u^fq)  for  problems  Qrj  will  go  to  infinity,  and  um(x)  will  eventually 
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produce  smaller  values  than  um(x j),  contradicting  the  assumption  that  the  latter 
is  a minimizer.  Thus  condition  (9.3.ii)  thus  holds  also,  and  Theorem  2 follows 
under  assumption  (9.1.i).  We  remark  that  the  above  argument  could  have  been 
based  on  a general  result  of  Witzgall,  Boggs,  and  Domich  [23]:  the  analytic  centers 
of  linear  constraint  systems  are  continuous  with  respect  to  changes  of  their  right 
hand  sides. 

Suppose  assumption  (9.1-ii)  holds,  and  recall  that  pm(z*)  = 0 by  (6.6).  For  r > 0 
by  Taylor’s  Theorem: 


(9.6) 


1 1 = 


uM{r) 


where  x(r)  is  a point  on  the  line  segment  [sc*,  sc(r*)].  We  will  first  show  that  the 

vector  dc(r)  in  that  problem  converges  to  zero  as  required  by  condition  (9.3.ii). 
By  (9.6),  and  since  B U C is  a partition  of  M,  vector  dc{r)  is  of  the  form 


(9.7)  dc{r)  = Tc(r)— — 

r 

with  the  matrix  Dc(r)  given  by 

Dc(r)  = Vgc[x(r)]T  - V gc[x(r)]TV  gB[x(r)}+T  gB  [x(r)]T 

Here  r > 0 must  be  sufficiently  small  for  the  matrix  VgB[r(r)\  to  have  full  rank.  It 
then  follows  that  its  pseudoinverse  converges  to  gB(x*)+ . Clearly,  x(r)  converges 
to  x*  - together  with  x{x)  - as  r > 0 tends  to  zero.  It  follows  by  continuity  that 

Dc(r)  - D'c  = Vgc(x-)T{I  - VffB(x*)+rVSB(s*)]. 


Recall  that  the  columns  of  V gc(x*)  lie  in  the  column  space  of  X7gB(x*): 

Vgc{x*)  = VgB(x*)VgB{x*)+Vgc{x*). 

Hence 


D'c  = Vgc(,x')1VgB(x')+TX7gB(x*)T  I - VSB(x*)+'rVSs(a:*) 


+T ' 


= 0. 


Thus  assumption  (9.1.ii)  implies  dc(r ) — > 0 in  view  of  (9.7). 

To  prove  the  remaining  condition  (9.3.ii),  we  assume  again  that  accumulation 
point  u is  the  limit  of  u(rj ),  Tj  >0,  j — 1, ...  , as  r3  — > 0.  Since 


xivj)  — x* 
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remains  bounded  for  rj  — > 0 by  (9.1.ii),  we  may  assume  that  it,  too,  converges  (or 
else  select  a suitable  subsequence  of  the  above).  By  (9.6), 

1 5Af[x(r3)] 

Mh)  = h = 9M[il)]  — — • 

The  reciprocal  of  um(t)  converges  since  VgM l^{rj)  converges  by  continuity  for 
j — > oo.  But  this  is  possible  only  if  the  limit  % is  positive.  The  conditions  of 
(9.3)  are  thus  met,  and  Theorem  2 is  proved.  (Note  that  (9.1.ii)  implies  M D Z. 
If,  furthermore,  the  primal  trajectory  limit  x * is  a strict  minimizer,  that  is,  if 
gi(x*)  = 0 only  if  i £ Z,  then  M — Z by  (6.6).)  □ 
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